Skip to content

feat: add agentic issue and PR processing#1503

Open
jescalada wants to merge 2 commits intofinos:mainfrom
jescalada:ai-automated-issue-and-pr-assessment
Open

feat: add agentic issue and PR processing#1503
jescalada wants to merge 2 commits intofinos:mainfrom
jescalada:ai-automated-issue-and-pr-assessment

Conversation

@jescalada
Copy link
Copy Markdown
Contributor

@jescalada jescalada commented Apr 15, 2026 •
edited
Loading

This PR adds automated issue triaging, PR description quality checks and diff vulnerability scanning.

Features

Issue triaging

Automatically processes freshly opened issues and:

  • Adds appropriate tags
  • Analyzes the n last issues to check for duplicates or related issues, provides reasons why they may be related/duplicate
    • n defaults to 100
  • Requests more info if the description insufficient

PR quality check

Automatically processes freshly opened PRs and:

  • Points new contributors to our CONTRIBUTING.md
  • Requests more info if the description is insufficient
  • Requests linking to an issue if not done
  • Checks if the PR description complies with the CONTRIBUTING.md, provides reasoning if not compliant

PR security review

Automatically processes freshly opened PRs, and also triggers when commenting "/security-review":

  • Looks at the PR diff and scans for vulnerabilities
    • Certain generated files are always excluded (*.lock, etc.)
    • The diff for a given is truncated down to 3000 chars to prevent bloating the prompt (it may contain generated files that weren't properly excluded)
Prompt details

Prompted specifically to detect:

  • Hardcoded secrets, tokens, passwords or API keys
  • Injection vulnerabilities (SQL, shell, template, etc.)
  • Insecure use of cryptography or hashing
  • Unsafe deserialization
  • Path traversal or directory traversal risks
  • Insecure direct object references
  • Missing input validation or sanitisation on user-controlled data
  • Use of known-vulnerable dependency versions (if visible in the diff)
  • Overly permissive file or network access

Examples

PR scanning/quality control

Sample vulnerable diff

PR security review on obvious vulnerable diff

Screenshots image image

Actual GitProxy PR opened on my fork (#1457)

PR security review on actual upstream PR

Screenshots image image image

Issue tagging

Screenshots image

To-do

  • Improve/clean up prompts
    • Shorten prompts to save tokens
    • Remove emojis
    • Standardize responses
  • Extract constants/make configurable through envs
  • Extract helpers into separate file, move agent scripts to more appropriate location

Notes

  • It'd be great to improve the issue triage to reproduce bugs and look for solutions. We could also review issues in more depth and assess feasibility or critique feature requests, etc.
  • I deliberately made the PR reviewer focus only on the security aspect, as we wouldn't want to raise eyebrows with full-blown AI reviews. That said, there's definitely a lot of added value in having AI process the diff before a human gets to look at it. I'm happy to implement a more sophisticated solution if we need it 🙂

@jescalada jescalada requested a review from a team as a code owner April 15, 2026 06:27
@netlify
Copy link
Copy Markdown

netlify bot commented Apr 15, 2026 •
edited
Loading

✅ Deploy Preview for endearing-brigadeiros-63f9d0 canceled.

Name Link
🔨 Latest commit cdd01cf
🔍 Latest deploy log https://app.netlify.com/projects/endearing-brigadeiros-63f9d0/deploys/69df36e968f7df0008fbee2f

@codecov
Copy link
Copy Markdown

codecov bot commented Apr 15, 2026 •
edited
Loading

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.21%. Comparing base (70d42d5) to head (cdd01cf).

Additional details and impacted files 
@@           Coverage Diff           @@
##             main    #1503   +/-   ##
=======================================
  Coverage   90.21%   90.21%           
=======================================
  Files          69       69           
  Lines        5511     5511           
  Branches      944      944           
=======================================
  Hits         4972     4972           
  Misses        521      521           
  Partials       18       18

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@jescalada